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Executive Summary 


This document is one of eight reports prepared to support the development of a new learning system, a 
development effort that is the first step in a major initiative undertaken by the Stupski Foundation. The 
Foundation endeavors to improve the life options of aU smdents, especially underserved urban youth of 
color, whom we refer to as “Our Kids,” by fundamentally redesigning the education system. 

This report was created coUaboratively by researchers from McREL with guidance from officers of the 
Smpski Foundation. Its purpose is to provide a summary review of the literature contributing to the 
definition of “college readiness.” The review aimed to identify the strategies, knowledge, behaviors, 
and skiUs needed before students are considered college ready upon graduation from high school. The 
definition of college readiness will provide members of a “Design Collaborative” team — consisting 
of researchers, practitioners, parents, and students — with a knowledge base from existing literature to 
support their efforts to develop a learning system that will bring aU students high levels of achievement 
and readiness for college. 

Research methodology 

McREL researchers, in collaboration with the Stupski Foundation, generated the following research 
question to guide this review: 

What strategies, knowledge, behaviors, and skills are essential before students are judged to be college 
ready at graduation from high school? 

To complete this report, a team of researchers at McREL reviewed a total of 230 books and articles, 
of which 214 were ultimately coded in a database. In some cases, this literature was based on surveys 
of post-secondary academics, business people, or other stakeholders whose opinions are valued in 
defining the expectations of a well-qualified college graduate. In other cases, the literature reported the 
findings from quantitative studies correlating certain student characteristics and elements of college 
success. Thirty-five articles were summarized and further categorized into four essential areas: cognitive 
strategies, content knowledge, academic behaviors, and contextual skills. These areas were taken from the 
Stupski Foundation’s document. Strategy and Program Overview July 2008, and were based on Conley’s (2007) 
previous work on college readiness. The complete search methodology is described in the Appendix. 

Key findings 

There were five main findings from the literature review. 

Content knowledge matters 

Across the literature, there was strong agreement that completion of a rigorous, comprehensive, 
college preparatory curriculum throughout one’s K— 12 education is a prerequisite for college readiness. 

A majority of authors identified content knowledge and the cognitive skills required to process that 
knowledge to be the most important factors in determining readiness for college. 
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There is lack of agreement about the specific content that matters most 

Despite agreement about the importance of completing a college-preparatory curriculum, there was disparity 
among authors as to what constitutes that curriculum. A deeper look at the curricular requirements of, for 
example, the America Diploma Project (2004) and the Knowledge and Skills for Student Success (Conley, 

2003a) shows only 25 percent agreement on English language arts content and only 57 percent agreement on 
mathematics content (Kendall, Pollack, Schwols, & Snyder, 2007) across two seminal college-readiness documents, 
each of which is guiding states and school districts across the country today. This leads to, at best, confusion for 
school system leaders and, at worst, discouragement and disenfranchisement for students. 

Metacognitive and self-system thinking are critical but under-emphasized or absent 

Many authors identified skills beyond the content domains as critical. Such skiUs include metacognitive thinking, 
or the ability to set learning goals and monitor one’s learning progress. Only one definition of college readiness 
explicitly addressed metacognitive skills as key for the mastery of important academic behaviors. More critical, 
according to the authors we studied, is students’ capacity to engage in self-system thinking, which includes the 
attitudes, beliefs, and emotions students hold about themselves as learners and about what they are learning. 
Indeed, students’ motivation is a key factor that authors identified as being perhaps most important, yet it was the 
aspect of student learning that was least addressed in the college readiness standards. 

College, career, and life readiness require similar knowledge and skills 

In the debate regarding the differences between college readiness and career readiness, most authors agreed that, 
in the 2P' century, the skills required for success in the high-skiUs workplace or in advanced training that leads to a 
rewarding career matched those required for success in college. 

Higher education is on the sidelines 

Although there was evidence in the literature of involvement by higher education in the determination of the 
knowledge and skills needed by entering college freshman (America Diploma Project, 2004; Conley, 2007), by 
and large, the evidence in this review shows higher education to be on the sidelines in this debate. Due to the 
lack of uniformity of American institutions of higher education, it is nearly impossible for the K— 12 community 
to engage with them at the policy level in order to ensure that a definition of college readiness will be uniformly 
accepted at any college or university Our Kids might choose to attend. 

Recommendations 

This report provides one primary recommendation with associated options for approaching the task. It is 
recommended that the Stupski Foundation develop its own definition of college readiness specifically designed for Our Kids. 
This recommendation is based on the following insights drawn from the literature review. 

The literature was clear regarding the importance of access to and completion of a rigorous and comprehensive 
college preparatory curriculum as a prerequisite for college readiness. Researchers also found a number of 
reports documenting the importance of self-system thinking, particularly the capacity for motivation, as well as 
metacognitive skills for a student’s ability to finish a preparatory curriculum and benefit from it in a college setting. 
The literature also was clear regarding the lack of opportunity many underserved students have to acquire these 
skills in the normal course of their development. Definitions that do not articulate these skills appear to assume 
that such skills are present in students who meet the stated criteria for college readiness. Although that may be a 
reasonable assumption for many students, it may not be reasonable for Our Kids. A definition to be used as the 
targeted outcome of the Learning System should make such skills explicit in order to ensure that they are taught, 
along with the important content knowledge. 
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This report offers three possible approaches to this task. 

Option 1: 

The Cafeteria approach, or choosing an existing definition that best suits multiple 
situations in which the Learning System will be applied 

The Cafeteria approach would offer districts working with the Stupski Learning System a menu of existing 
definitions from which to choose with some modifications to ensure that the essential content for Our 
Kids is emphasized. This approach recognizes the significant and empirically based efforts that already 
have been made in identifying rigorous, subject-area standards, which include content associated with 
student success in post-secondary institutions. In addition, it allows districts to choose an approach 
based on their own context. The challenge of the Cafeteria approach is establishing a set of guidelines 
to ensure that adopted programs include those attributes of the self-system that have been identified as 
critical in helping Our Kids succeed. 

Option 2: 

The Conversion approach, or choosing from among the extant definitions the one 
that will best meet all possible situations 

The Conversion approach means directing aU districts that are working with Stupski to adopt, or convert 
to, the same approach. For example, all districts might become International Baccalaureate (IB) schools, 
or all districts might adopt the Knowledge and Skills for University Success (KSUS) standards and a 
program that supports them. This approach assumes that the foundation can identify a single, existing 
program that best meets its criteria and then supplement it, where needed, to ensure that students will 
receive the support they need in developing metacognitive and self-system thinking. The advantage 
of this approach is consistency across the system and the associated economies of scale. Also, the 
districts may receive support from the program provider in terms of resources and in making the case 
to others — in the community as well as in higher education — that the program adequately prepares 
students for college. The disadvantage is that any contextual considerations of individual districts might 
be ignored in the adoption process. 

Option 3: 

The Composition approach, or, by drawing from the extensive work that has gone 
before, building a definition that best suits the goals and objectives of the 
Learning System as it is envisioned 

In the Composition approach, college readiness is defined by Stupski for its partner districts, and the 
curriculum and program of instruction are custom-built or adapted from extant curricula to ensure 
that students will meet the requirements of the new definition. The label for this approach. Composition, 
reflects identifying and bringing together standards and programs from across organizations. The work 
begins by identifying the best of current thinking across all academic subjects, synthesizing content 
specifics across standards, and identifying the strengths of each, with attention to the cognitive strategies 
that ensure student success. The work reviews the set of academic behaviors in order to establish a 
common language, making clear the types of skills and capacities that define the successful student. 

The advantage of this approach is that it is innovative, yet capitalizes on work already done, and that it 
realizes economies of scale by developing common goals across the Learning System and ensures that 
every curriculum used across every district and grade helps prepare students for college. In this approach, 
any curriculum could be reviewed for its suitability in promoting aU aspects of the Stupski-established 
definition, although not every curriculum may be acceptable. 
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There are, of course, significant challenges to this approach. The fact that districts must meet state requirements 
remains an issue, unless all Stupski schools become charter schools and negotiate the means for accountability 
state by state. This effort also requires considerable time, effort, and expense. 

Final thoughts 

Defining college readiness for aU students in the 2D' century in America today is a formidable task. This report 
identifies strong areas of agreement in the literature — the importance of content knowledge and of metacognitive 
and self-system thinking — but also highlights the paucity of definitions and associated programs that successfully 
articulate all attributes. Thus, the Foundation should consider adopting one or more of the three approaches 
described here, with the expectation that the depth and scope of the definition they develop will vary depending 
on the approach or approaches they endorse for Our Kids. Ultimately, the Learning System and Our Kids will 
best be served by a definition that is generated from the same spirit and fervor upon which the entire mission of 
the Foundation is based. 


4 


Introduction 


Purpose of this document 

This document is one of eight reports prepared 
to support the development of a new learning 
system, a development effort that is the first 
step in a major initiative undertaken by the 
Stupski Foundation. The Foundation endeavors 
to improve the life options of aU students, 
especially underserved urban youth of color, by 
fundamentally redesigning the education system. 

This report was created coUaboratively by 
researchers from McREL and officers of the 
Stupski Foundation. Its purpose is to provide 
members of the Design Collaborative team with a 
review of key findings from the existing literature 
regarding critical research questions related to the 
College Readiness component of the Learning 
System and to offer recommendations for the 
development of this component. Together, the 
reports cover these topics: 

• Assessment 

• Curriculum 

• Pedagogy 

• Student Supports 

• Systems Diagnostics 

• Leadership 

• College Readiness 

• Our Kids 

The first section of this report provides salient 
findings that emerged from the literature review. 
The second section offers a discussion of the 
findings along with one primary recommendation 
and three implementation options for how the 
Design Collaborative might proceed. A brief 
concluding discussion follows. Summaries of the 
studies and literature reviewed for this report were 
provided as separate documents. 


About the Learning System 

The Learning System is the product of the 
Stupski Foundation’s extensive examination of 
research, best practices, and theories of action 
for improving education opportunities for all 
children. It is deeply rooted in the Foundation’s 
mission to foster innovation in public school 
systems so that all students graduate ready for 
college, career, and success — as well as the notion 
that the United States’ education system, in its 
current state, is incapable of accomplishing this 
goal. As stated on the Foundation’s Web site, 

“The basic components of what public education 
systems need to teach aU students to world-class 
standards, particularly those students for whom 
public schools are their only option, do not exist 
in any coherent, accessible or evidence-based 
way” (Stupski Foundation, n.d.). 

Thus, the Foundation has focused its 
philanthropic efforts on supporting the 
“fundamental reinvention” of the American 
system of public education into one that prepares 
aU children for the challenges of Ufe, career, and 
citizenship in the 21 century. To accomplish this 
objective, the Foundation launched a multi-year, 
cross-sector collaboration among researchers and 
practitioners from inside and outside education 
to develop a new and comprehensive learning 
system. In its June 2008 Strategy and Program 
Overview, the Foundation posited that this system 
includes seven components, shown in Figure 1 
(see p. 6). The indicators of success are dependent 
on a definition of college readiness, which is 
addressed in this report. Although Our Kids is not 
an explicit component of the Learning System, it 
is the foundation for the work the foundation is 
committed to in the education sector. As such, the 
populations of students of color and students of 
poverty warranted a separate report. 
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Systems Components 


Figure 1: The Learning System 


I ndicators of Success: 

Cognitive Strategies, Content Knowledge, Academic Behaviors, Contextual Skills 

The "dashboard" establishes the student achievement outcomes and performance standards — the measures 
of college-career-citizenship readiness — that will provide evidence of an effective learning system. 

Systems Diagnostics: State, District, School 


Systems diagnostics measure the extent to which states, districts and schoois have established the 
systems, services and supports essential to college readiness for all students. 


Leadership/ Human Capital 





Capacity and Culture to Deliver the Learning System 

Leadership roles, responsibilities, skills and behaviors essential to creating the 
conditions critical to the effective implementation of the Learning System. 


Curriculum 

The college readiness core 
curriculum identifies the 
learning progression of 
cognitive and affective 
skills that students must 
acquire at each step of 
learning to be ready for 
snccess at the next level, 
ultimately exiting schools 
ready for success in college, 
career and citizenship. 


Assessments 

Real-time performance- 
based assessments that 
monitor student 
performance and growth 
and provide quick 
feedback cycles. 


Pedagogy 

Instructional practices that 
effectively deliver 
advanced content and 
enable teachers to tailor 
their instruction to the 
diverse learning needs 
within their classrooms. 


Supports 

I nstructional 
interventions and 
socioemotional 
supports that help ensure 
Itet student achievement 
is on the right trajectory. 


About "Our Kids" 

The Stupski Foundation is committed to addressing the academic needs 
of underserved populations, in particular, students who are of color and in 
poverty (which comprises 42% of African American students and 37% of 
Hispanic students) (Duncan & Magnuson, 2005). Despite a dramatic rise 
in minorities enrolling in college (a 50% increase from 1995—2005), fewer 
minorities appear to be graduating. As shown in Figure 2 (see p. 7), in 2006, 
fewer minorities aged 25-29 reported having obtained an associate degree 
or higher than their older peers (aged 30 and over) (American Council on 
Education, 2008). This trend marks an important reversal in advances in 
educational opportunities for minorities and may mark the first time in 
history that a generation of students has demonstrated less educational 
attainment than its predecessors (American Council on Education, 2008). 




Overview of methodology 

This study was conducted to inform the specific 
needs of the Design Collaborative, a cross-sector 
group of researchers, practitioners, and designers 
from inside and outside education, to “define, 
develop and continually improve” (Design 
Collaborative, n.d.) aU of the components of the 
Learning System. As such, this study does not 
represent an exhaustive review of aU literature 
relating to a definition of college readiness. 

Rather, the search was conducted, and the 
quality of literature evaluated, using an inclusive 
approach that focused on relevance to meeting 
these specific needs (see Appendix for a thorough 
discussion of methodology). Following is an 
analysis of selected literature deemed best suited 
to the study’s objectives. 

McREL researchers followed a five-step process 
for translating findings into recommendations. 

Step 1: Identification of key hypothesis 

After conducting an initial survey of relevant 
literature, McREL researchers, along with 
Stupski Foundation staff members, identified the 
following hypothesis to guide the literature review 
for the College Readiness component: 

If students implement certain cognitive 
strategies, possess defined levels of content 
knowledge, exhibit specified academic behaviors, 
and perform appropriate contextual skills, they 
will be able to perform satisfactorily in college. 

Step 2: Identification of research questions 

McREL researchers, in collaboration with Stupski 
Foundation staff members, generated this 
question: 

What strategies, knowledge, behaviors, and skills 
are essential before students are judged to be 
college ready at graduation from high school? 


Figure 2: Percentage of U.S. adults with 
associates degree or higher, 2006 

70 
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Source: American Council on Education, 2008 



Step 3: Literature search 

Initially, the college readiness search began 
with a gathering of research reports and 
information provided by seminal coUege- 
readiness organizations such as Achieve, ACT, 
and College Board. Researchers also explored 
the Web sites of organizations known to provide 
quality educational research such as WestEd, lES, 
Fordham Institute, and the National Governor’s 
Association. Most searches were limited to articles 
and reports more recent than 1998. 

Once seminal documents were gathered, 
researchers began to search academic databases 
for articles and additional research relating to 
the definition of college readiness and cognitive 
strategies, context knowledge, academic behaviors, 
and contextual skills needed to prepare students 
for college. The databases used to conduct the 
initial searches were ERIC, Proquest, Education 
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Research Complete, and Academic Search Premier. Although the search 
centered on materials addressing an overview of college readiness, 
researchers also took a special interest in documents that addressed children 
of poverty. The search focused on documents with claims supported by 
evidence rather than those that advocated an unsupported opinion. Sources 
were searched by the following keywords: 

• College bound students 

• College preparation 

• College readiness 

• College ready 

• Low income 

• Nontraditional education 

• School readiness 

• Urban population 

• Workforce readiness 

Articles identified were retrieved and skimmed with particular attention 
to research methods, outcomes, and recommendations for future study 
due to gaps in knowledge. Throughout the coding of the documents, 
researchers identified and retrieved additional articles and reports that were 
frequently referenced or addressed an unexplored facet of the definition. 

As the direction of the report began to take shape, the search included 
articles suggested by internal experts Monette Mclver and John Kendall. 

As a result of discussions regarding the interconnectivity of workforce 
readiness and technical education with the concept of college readiness, the 
search expanded to include these topics. Also, press releases from various 
organizations were scanned for the latest relevant materials. In total, 230 
articles were retrieved, and 214 of those were recommended for further 
coding. Ultimately, the team summarized 35 articles related to College 
Readiness, which are in a separate annotated bibliography. 

Step 4: Identification and cataloging of findings 

The research team cataloged findings from the summarized articles into 
four essential areas: Cognitive Strategies, Content Knowledge, Academic 
Behaviors, and Contextual Skills. These areas were taken from the Stupski 
Foundation’s document. Strategy and Program Overview July 2008, and were 
based on Conley’s (2007) previous work on college readiness. 

Step 5: Generation of recommendations 

In the final phase, research team members collectively reviewed key findings 
from the literature review in light of the following questions: 

• What are the critical unmet needs related to this component of the 
Learning System? 
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• What is missing in current practices within this 
component of the Learning System? 

• What is working and why? 

• What is «o/ working and why? 

• What are the biggest misalignments between 
research and current practice? 

• What things should educators do differently in 
light of the research findings? 

• Where is the knowledge base too inconclusive 
to guide education innovation? 

• Where is more research needed to advance 
practice? 

Responses to these questions were synthesized 
into recommendations, presented here as options 
for further action. 

Overview of the literature 
base examined 

The Stupski Foundation’s document, Strategy 
and Program Overview July 2008, provided an 
initial organizing framework for the literature 
search based on Conley’s (2007) previous work 
on college readiness. Relevant literature was 
identified and categorized into four essential 
areas: Cognitive Strategies, Content Knowledge, 
Academic Behaviors, and Contextual Skills (see p. 
11). These categories accommodated the variety 
of knowledge, behaviors, strategies, and skills 
identified in the research literature. 

A key assumption for this project was that there 
is a set of strategies, knowledge, behaviors, and 
skills that students should possess in order to be 
college ready upon graduation from high school. 
This key assumption led to the development of 
one foundational question that guided the search 
process: What strategies, knowledge, behaviors, and 
skills are essential before students are judged to be college 
ready at graduation from high school? 

Several authors and organizations were identified 
during the search process whose work is 
highlighted in this report: 


• ACT is a non-profit organization with a 
mission to help people achieve education 
and workplace success. Though perhaps best 
known for its college-ready assessments, 

ACT has conducted research in college- and 
workplace-readiness. 

• The American Diploma Project (ADP), a 
partnership of Achieve, Inc., The Education 
Trust, and the Thomas B. Fordham 
Foundation, defines English and mathematics 
benchmarks of college and workplace success 
for high school graduates. ADP’s 2004 seminal 
report, Peady or Not: Creating a Nigh School 
Diploma That Counts, is used as the basis of 
high school reform efforts in 34 states. 

• David T. Conley is the director of the Center 
for Educational Policy Research at the 
University of Oregon. Dr. Conley directed the 
Standards for Success project, which defined 
knowledge and skills students need to succeed 
in entry-level university courses. Dr. Conley’s 
2007 report. Toward a More Comprehensive 
Conception of College Readiness, prepared for the 
Bill & Melinda Gates Foundation, has been 
extensively used by the Stupski Foundation to 
guide this work. 

• Steven B. Robbins is lead author of two 
reports focusing on the psycho-social aspects 
of college readiness. Mr. Robbins’ and his 
coauthors’ contributions are notable in an 
area marked for its importance to college- 
readiness but relatively lacking in high-quality 
information. Mr. Robbins is in the Research 
Division at ACT. 

• The College Board is best known for its 
programs, the SAT and the Advanced 
Placement (AP) programs. The College Board 
also facilitates original research to support the 
transition from high school to college. 

Articles that specifically included explicit 

definitions of college readiness were identified 

for further analysis. This select group of the 
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coded articles was then summarized to capture key points about, and salient 
contributions to, the definition of college readiness. These summaries were 
compiled into a separate annotated bibliography. Summarized literature 
was broadly categorized into five types of literature: research studies 
(quantitative and qualitative), research reviews, program descriptions, 
position papers (papers that advocated a particular point of view), and other 
(books in popular press). As shown in Figure 3, the majority of college 
readiness literature was composed of research studies and position papers. 


Figure 3: Types of literature summarized 



■ Research Study 

■ Position Paper 

■ Program Description 

■ Research Review 

■ Other (Books) 


Introduction to the concept map 

The following concept map (see Figure 4, p. 1 1) illustrates the hypothesized 
relationship of the essential areas and the research question. The remainder 
of this report discusses research insights about the four essential areas of 
the college readiness definition identified by Conley and adapted by the 
Foundation, additional attributes identified by other researchers, and options 
that more clearly describe the target outcomes of a learning system for 
Our Kids. These findings and options are foundational to the successful 
fulfillment of aU components of a learning system designed to prepare Our 
Kids, and all kids, for college. 
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Figure 4: Concept map 


Primary Research Question 

What strategies, knowledge, 
behaviors, and skills are essential 
before students are judged to be 
college ready at graduation from 
high school? 

V J 


Cognitive Strategies: 

Pproblem-solving techniques 
and higher-order thinking 
skills required to thrive in our 
increasingly complex world. 


Content Knowledge: 

Fundamentals students 
should know in order to have 
a sound knowledge base 
and be full participants in our 
democracy and economy 


Academic Behaviors: 

Skills and habits of mind that 
enable individuals to pursue 
and acquire knowledge and 
goals. 


Contextual Skills: 

Ability to apply knowledge 
in a variety of contexts and 
use contextual clues to make 
meaning with new words, 
problems and settings 
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Findings 


Content knowledge matters 

The seminal documents reviewed for the literature 
search (ACT, 2005; American Diploma Project, 
2004; Conley, 2007; The College Board, 2006a; 
The College Board, 2006b), either explicitly 
or implicitly defined as college -ready those 
high school graduates who are able and likely 
to succeed in college without the need for 
remediation. Much of the relevant literature 
addressed college readiness as the accumulation 
of a set of cognitive strategies and content 
knowledge. There was general agreement that 
students must be exposed to rigorous content 
and must have opportunities to develop complex 
cognitive processes. 

Content knowledge 

A wide range of the college readiness articles 
(Achieve, 2008; ACT, 2005; Adelman, 1999; 
Adelman, 2006; AUen & Sconing, 2005; American 
Diploma Project, 2004; Conley, 2003b; Conley, 
2007; Haycock & Huang, 2001; Kuh, Kinzie, 
Buckley, Bridges, & Hayek, 2006; Martinez 
& Klopott, 2005; The College Board, 2006a 
& 2006b) reported evidence that high school 
graduates who undertake a four-year regimen of 
rigorous coursework in mathematics, science, and 
English are more likely to succeed in college than 
those who do not. 

It is not enough for high school graduates simply 
to pass through these courses in some sort of 
pro forma sequence and expect to do well in 
college. Rather, they must meet high academic 


expectations and accomplish a series of courses 
with increasing degrees of difficulty. Kuh et al. 
(2006) found that 87 percent of students who 
completed four high school years of mathematics, 
science, and English stayed “on track” to graduate 
from college, compared with 62 percent of those 
students who did not complete that panoply of 
high school coursework. With respect to Our 
Kids, Adelman (1999) claimed that the high 
academic intensity and quality of high school 
curricula influenced college degree completion 
to a far greater degree for African American and 
Latino students than for white students. 

One of the salient conclusions among the authors 
of this literature is the significant role of higher 
level mathematics in preparing high school 
graduates for college-level courses. High school 
students who master Algebra I & II, geometry, 
trigonometry, pre-calculus, and calculus— especially 
the latter three-have considerably improved 
odds of completing a bachelor’s degree. Further, 
the American Diploma Project (2004) and The 
College Board (2006b) stressed the increasing 
importance of proficiency in statistics and 
probability in becoming an informed citizen of 
the 2L' century. Kuh et al. (2006), in particular, 
argued vigorously that, in order for high school 
students to perform well academically in high 
school, particularly on the mathematics courses, 
they also must experience rigorous academic 
preparation in grade school and middle-school — 
especially in mathematics and reading* 


'After the College Readiness database had been prepared and parsed, its articles summarized, and drafting of the final 
report begun, a study was published (and is now widely reported) suggesting that a 1997 Chicago Public Schools initiative 
requiring all of its 9* graders to enroll in algebra may have failed to heighten mastery of algebra or to encourage students 
to take more advanced mathematics courses (Allensworth & Nomi, 2009). In light of the recommendations included in 
the college-ready literature urging high school students to take higher mathematics courses after algebra II, the factual 
circumstances of this study deserve further examination to determine what caused the results reported. At least two issues 
worthy of further analysis are the extent to which both [to help make clear what the two issues are] student supports and 
classroom instruction may have influenced the student outcomes found by these researchers. 
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Contextual skills 


Contextual skills, expressed by the Foundation as the ability to apply 
knowledge in a variety of contexts, appeared with varying levels of description 
and emphasis across the literature related to college readiness. The American 
Diploma Project, for example, described contextual skills in terms of the 
content knowledge related to “associated workplace tasks” and “associated 
post-secondary assignments.” The College Board defined contextual skills in 
terms of their significance at six different performance levels. 

Cognitive strategies 

Cognitive strategies— such as problem-solving techniques and higher order 
thinking skills— were also identified in the literature as important for the 
college readiness of high school graduates (Achieve, 2008; Barton & 
Lapointe, 1995; Conley, 2007; HiU, 2008; Levine, 2007; Organisation for 
Economic Co-operation and Development [OECD], 2005; Partnership 
for 2P' Century Skills, 2006), although there is little agreement on how to 
describe those skills. Barton & Lapointe (1995), for example, identified 
communication, critical thinking, and abstract reasoning as influential 
faculties for college success. Conley (2007) articulated “key cognitive 
strategies” necessary for college success, which he states are “always 
developed within the ways of knowing a particular content area” (p. 16). 
These cognitive strategies are manifest through, among other things, 
intellectual openness and curiosity, as well as through reasoning and analysis. 
Levine (2007) wrote of a “cognitive backpack” of soft skills necessary 
for college and career success, including “interpretation, instrumentation, 
interaction, and inner direction” (p.l6). The North Central Regional 
Educational Laboratory (2003), in discussing skills required for 2P' century 
tasks, posited that there are four categories of skills essential for success: 
digital-age literacy, inventive thinking, effective communication, and high 
productivity skills. 

The cognitive skills needed for both 2P'-century tasks and college readiness 
were qualitatively described with varying degrees of precision across a broad 
continuum of literature, including best-seUing authors. For example, Thomas 
Friedman (2006) and Daniel Pink (2005) expressed their views on attributes 
students must exhibit to engage 2P' century challenges. Friedman stressed 
the importance of the ability to learn how to learn, individual passion and 
curiosity, and skillful personal interaction; Pink defined six new “senses” 2P' 
century students must acquire — design, story, symphony, empathy, play, and 
meaning — ^which include affective qualities. 


14 


There is lack of agreement about the 
specific content that matters most 

Definitions vary in scope and detail 

The organizations advocating comprehensive 
definitions and associated college preparation 
programs varied in scope and level of detail 
when defining content knowledge. For example, 
The College Board has defined college readiness 
through a set of standards in English language 
arts that includes six levels of performance and 
through objectives for mathematics that range 
from middle school to Algebra II and pre-calculus. 
ACT has described multiple levels of proficiency 
for both English language arts and mathematics 
in college readiness standards stratified by score 
ranges. These ranges have been established based 
upon the test performances of middle school and 
high school students who later proved themselves 
successful in post-secondary work. 

In contrast, the standards from ADP and 
Knowledge and Skills for Student Success (KSUS) 
were summary descriptions developed principally 
from surveys of professors in higher education 
and/or industry representatives of the high-skiUs 
workplace. The descriptions of student knowledge 
and skills represent what students should acquire 
by the end of their high school experience to be 
college-ready. 

Especially problematic in understanding “college 
readiness” is the differing expectations these 
organizations have for students. Although both 
KSUS and ADP identified English language 
arts and mathematics as significant content for 
post-secondary work, one study comparing the 
standards from these organizations showed they 
shared just 25 percent of the English language 
arts content and 57 percent of the mathematics 
content (Kendall, Pollack, Schwols, & Snyder, 
2007). A contributing factor to this discrepancy 
was the nature of the specifics emphasized in the 
reports — the emphasis on literature in the KSUS 
set of standards, for example, as opposed to the 


importance of oral presentation in the ADP set. 
This difference, however, also indicates that the 
knowledge and skills required for college readiness 
(KSUS) might not be identical to those needed 
for post-secondary and the high-skiUs workplace 
(ADP). Yet, as troubling as such apparent lack of 
agreement may be, it is a welcome alternative to 
vague and nearly useless prescriptions for college 
readiness, such as “students should master algebra 
II.” It is clear that merely identifying requirements 
by course name is no solution (Achieve, 2004). 

Despite these discrepancies, there appears to be 
agreement that rigorous and specific standards 
in the English language arts and mathematics 
are essential to college success. Yet, it is not clear 
whether these organizations consider the mastery 
of other academic content important, as well. Of 
the four organizations (ADP, KSUS, SAT, The 
College Board) that have articulated standards 
for college readiness, just two address science 
standards — KSUS and SAT — and only one, 
KSUS, addresses standards for the arts, second 
languages, and the social sciences. 

Finally, to add to the confusion, the literature even 
lacks common agreement about the term “college 
ready.” Kuh et al. (2006), for example, found that 
“college ready” meant ready to be accepted at a 
post-secondary institution in one case and ready 
to earn a baccalaureate degree in another, as well 
as acquisition of specific content knowledge 
at certain points in time and more general 
acquisition of non-academic skills at others. Kuh 
et al. found that the direct interests of those 
defining the term influenced the elements that 
were added or emphasized, thus exacerbating 
inconsistencies. 

"Common Core" standards a de facto 
definition? 

The No Child Left Behind Act left it up to states 
to determine their own content standards and to 
also set their own bar for what should constitute 
proficiency against those standards. As a result. 
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the nation now has 50 different sets of standards and 50 different levels 
of proficiency. This has led to a vast discrepancy among state content 
standards and highly variable results of state standardized assessments 
compared with the National Assessment of Educational Progress (NAEP) 
examinations, further complicating definitions of college readiness. The 
Common Core State Standards Initiative of the National Governors 
Association, the Council of Chief State School Officers, and Achieve, Inc., 
which seeks to get states to adopt a common set of national standards 
(Hoff, 2009), may signal a movement toward reconciliation among the 
variety of definitions of the content students should know in order to be 
college-ready. The draft standards in English language arts and mathematics 
were released for comment on September 21, 2009.^ 

Metacognitive and self-system thinking are critical but 
under-emphasized or absent 

“Academic behaviors” — in this study defined as the critical student 
behaviors that depend upon the effective use of metacognitive and self- 
system skills — appeared less commonly in the literature, and often were not 
clearly distinguished from the types of cognitive strategies described in the 
previous section. Metacognitive skills relate to students’ attention to their 
own learning as well as their goals for learning. Self-system skills, as defined 
in a recent taxonomy of educational objectives (Marzano & Kendall, 2007) 
encompass the “interrelationship of attitudes, beliefs, and emotions students 
hold about themselves as learners and are the prime determiner in the 
motivation they bring to a task” (p. 12). Conley most clearly distinguished 
academic behaviors from cognitive strategies “by the fact that they tend 
to be more completely independent of a particular content area” (p. 16). 

He described academic behaviors as primarily study skills, such as time 
management and effective note taking; and self-monitoring skills, such as 
persistence through difficult tasks, the ability to diagnose one’s own lack of 
understanding, and the ability to devise plans to achieve competency. Such 
skills are commonly associated with the metacognitive, in that they focus on 
student’s attention to their own learning and learning goals. 

The two elements of academic behavior — metacognitive and self-system 
thinking — received disparate attention in the literature. Some aspects of 
metacognition appeared very rarely and only indirectly in most standards 
documents, such as the College Board standards in the language arts, which 
focus on students identifying a goal for reading. Conley’s definition was 
the only work to attend specifically to metacognitive skills. No definitions 
of college readiness identified in this search, however, clearly included 
self-system processes; rather, most definitions only indirectly alluded to 
the importance of student motivation. This is not especially surprising 


“See www.commoncore.org for the draft standards. 
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because self-system processes, such as examining 
the importance of a task and one’s emotional 
response to it, “probably represents a level of 
introspection and conscious thought not normally 
engaged in” (Marzano & Kendall, p. 60). Yet, 
research suggests that students’ attention to 
motivation is critical. 

Adeknan (2006) concluded that individual student 
responsibility stands out as an important qualitative 
feature of college preparedness, saying that 
students “are partners in their own education fate” 
(p. xxvi). Similarly, anecdotal reports of success 
stories of students “at risk for poor performance” 
identified as a critical feature students’ 
understanding that they are responsible for their 
own success (Goldberger & Bayerl, 2008). 

More specifically, two studies (Robbins et ak, 

2004; Robbins, Allen, Casillas, Peterson, & Le, 
2006) found that motivational constructs were 
associated with college performance, and that 
academic discipline and general motivation were 
strongly related to academic outcomes. In a third 
study, Levine (2007) advocated that, in addition 
to traditional academic and technological skills 
required for college- and life-readiness, students 
should be equipped with a “cognitive backpack” 
of skills, among them a developed inner direction 
that assures that one’s decisions are informed 
by authentic insights into one’s own strengths, 
weaknesses, and affinities. 

A concept closely related to the “self-system” 
was further identified by Friedman (2006), HiU 
(2008), and OECD (2005) as a critical element of 
college readiness and educational success. HiU, in 
particular, in his description of the International 
Baccalaureate program, described a learner who is 
open-minded, balanced, an inquirer and a thinker. 
The various attributes described by these authors 
may be taken together to define a set of skiUs 
beyond the academic that a fully coUege-ready 
student must possess. 


The lack of attention to metacognition and 
the self-system is not only common among 
definitions of coUege readiness. In their study 
to identify thinking and learning skills described 
in standards documents from a variety of state 
departments of education, national subject-area 
organizations, and from organizations concerned 
about adequate student preparation for post- 
secondary work, KendaU et al. (2008) found 
that general cognitive skills — such as recaU and 
comprehension — ^were commonplace across all 
documents and subject areas. However, more 
complex cognition, classified as “knowledge 
utUization” and including such processes as 
decision making and investigation, were not weU- 
represented in the standards. Expectations that 
students should engage in metacognition were 
even less commonly addressed in the standards; 
any references to self-system thinking — “the 
attitudes, beUefs, and emotions that students hold 
about themselves as learners and about what they 
are learning” — ^were less common still (p. 3). To 
reiterate, when these more complex processes 
related to self-system thinking were addressed 
in the literature, they were presented as equally 
important as content knowledge in definitions of 
college readiness. 

Our Kids fall short 

The literature was clear that there is no substitute 
for completion of a rigorous college preparatory 
curriculum and attainment of the associated 
content knowledge and cognitive skills if one is to 
be truly prepared for the academic requirements 
of higher education. However, the literature also 
revealed a number of areas in which the education 
of Our Kids falls short.^ For example, even before 
they apply to college. Our Kids lack access to 
higher mathematics and otherwise rigorous high 
school curricula due to the misconception among 
K-12 practitioners that Our Kids don’t possess the 
intellectual capacity to master an intense academic 


^More detailed information about issues related to the elements of a college-preparatory curriculum and how to 
overcome the historical impediments to accomplishing its requirements for Our Kids is contained in the other component 
reports in this series. 


Stupski Foundation's Learning System / College Readiness 17 


sequence of high school curriculum, particularly higher level mathematics 
courses (Achieve, Inc., 2008; Goldberger & Bayerl, 2008; Kuh et al, 2006). 

College, career, and life readiness require 
the same skills 

Many of the articles reviewed emphasized the importance of a rigorous high 
school education in preparing not only for college but also for challenging 
and rewarding 21®' century careers. Typical of this aspect of the literature 
was Achieve’s (2005) finding that employers report that high school 
graduates they hire for high-skilled, well-paying jobs need the same skills 
and knowledge that colleges and universities claim enrolling students should 
possess. ACT (2006) reported similar findings in its analysis of workforce 
readiness. Haycock and Huang (2001) noted that graduating students are 
currently not fulfilling the literacy and mathematics demands of employers, 
and that the skills required in 70 percent of new jobs will require at least 
some post-secondary education. Similarly, The College Board (2006ab) 
asserted that the knowledge and skills in mathematics and English that are 
needed for college success are the same as those required for entry-level jobs 
that can lead to a rewarding career. 

Higher education is on the sidelines 

Though an agreed-upon definition of college readiness is necessary in 
creating a system that prepares aU students for post-secondary success, it 
must also be noted that such a definition can only go so far without agreed- 
upon expectations from institutions of higher education. The perspective 
of these institutions was noticeably “missing in action” in the literature on 
college readiness. Though certain projects specifically attempted to collect 
input from higher education (American Diploma Project, 2004; The College 
Board, 2006ab; Conley, 2003a), litde has been done to ascertain the degree 
to which the reported perspectives of higher education are reasonable and 
uniform. Olson (2006) found that it is unknown if perspectives of college 
professors on necessary “college-ready” qualities reflect the professors’ 
insights on the characteristics of a “perfect” student or insights on what 
is needed to prepare students to complete coursework with a passing 
grade. Olson also found inconsistencies in requirements for admission 
and placement without remediation. In most states, individual colleges and 
universities select their own placement tests and their own standards for how 
well students must perform to avoid remediation, sending “mixed messages” 
to incoming students (p.4). This leaves high schools to develop their own 
graduation criteria, which may or may not be aligned with the expectations 
of higher education. Without agreement from at least some colleges about 
what they will and will not accept as college-level work, on a consistent and 
reliable basis, and throughout a student’s four-year college experience, the 
architects of the Learning System will have difficulty constructing a learning 
progression for Our Kids that results in the desired outcome. 
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Discussion & Recommendations 


Discussion 

This study followed the practice of Kuh et al. 
( 2006 ) by not limiting examination to a specific 
narrow definition of college readiness; rather, 
it included all studies that defined elements 
of college readiness. The findings, therefore, 
represent a broad sweep of existing definitions 
of college readiness. Perhaps it is because 
of this that the inconsistencies are so readily 
apparent. Whether the literature offers definitions, 
or elements of definitions related to content, 
metacognition, or the self-system, there is Utde 
agreement among researchers as to the specifics 
within the definitions they assert. A Learning 
System requires concrete details in the areas of 
standards and curriculum. Many of the extant 
definitions do not provide these specifications; 
others provide them but in too little detail to 
be of use; and those that do provide them are 
sufficiently mismatched with one another so as to 
offer confusion, rather than guidance, to system 
developers trying to learn from previous work. 

Our findings reveal that the majority of 
definitions focus on the importance of content 
knowledge. The area of skills and attributes 
that support essential academic behaviors is far 
less well developed in the extant literature base. 
However, given the level of content knowledge 
required by most definitions, it does not appear 
reasonable to conclude that metacognitive and 
self-system thinking could be absent in a learner 
who succeeds at attaining that knowledge. It 
may be that content-heavy definitions of college 
readiness include a tacit assumption that college- 
ready students already possess these more 
reflective skills and, therefore, they need not be 
explicated in the definition. 


Although there are cases where such an 
assumption may be valid, the authors question 
whether this applies to Our Kids. Many children 
have opportunities to develop such skills outside 
of school in their interpersonal interactions, life 
experiences, and upbringing. Rich developmental 
experiences facilitate the growth of a child’s 
capacities to learn how to learn and to develop 
the intrinsic motivation necessary to support 
academic success, so that schooling can focus 
on content and cognition. For Our Kids, 
developmental experiences outside of school may 
be absent, requiring them to be included in the 
Learning System itself if Our Kids are to acquire 
these associated attributes. 

There are many impediments to the development 
of healthy perspectives for children who live in 
poor, urban environments. In such environments, 
the rich and stimulating developmental 
experiences common for middle-class youth are 
often replaced by basic survival experiences, rife 
with developmentaUy inappropriate stimulation.'* 
While a student with fully developed capacities 
related to self-understanding may well be able 
to access and realize a content- and cognitive- 
laden definition of college readiness, a less 
well-equipped student may not. Further, if the 
development of these capacities is not explicitly 
articulated in the definition of college readiness, 
they are unlikely to be present and developed in 
the associated Learning System and, as a result. 
Our Kids’ opportunities to become college-ready 
are diminished. 

When students are recognized as academically 
deficient, too often the current education system 
tries to force content-heavy expectations on them 
by replacing intellectual rigor with rote learning. 


■*See the report on the Our Kids component for more on the challenges to proper psycho-social development. 
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interaction with scripted response, and intrinsic engagement with external 
mandates. When the defined outcomes of a content-driven Learning System 
fail to acknowledge the need to develop metacognitive and self-system 
thinking, the system is left with inequities between students who are, and 
who are not, prepared to succeed. Too often, as described by Beers (2009), 
those not prepared to succeed are relegated to schooling that not only is 
less demanding in terms of content, but also eliminates the educational 
experiences necessary for the development of the capacities necessary for 
college readiness — thus perpetuating the cycle. 

Promising practice 

An integrated approach to college readiness may be exemplified currently in 
the modern International Baccalaureate (IB) Program. Although there was 
no evidence in the literature of the IB program serving Our Kids, on a broad 
scale, the program does intentionally incorporate self-system outcomes into 
its education mission, and that intentionaUty has resulted in a program that 
expUcitiy develops those skills in its students. 

IB programmes promote the education of the whole person, emphasizing 
intellectual, personal, emotional and social growth through aU domains of 
knowledge. By focusing on the dynamic combination of knowledge, skills, 
independent critical and creative thought and international-mindedness, 
the [International Baccalaureate Organization] espouses the principle of 
educating the whole person for a life of active, responsible citizenship. 
(International Baccalaureate Organization, 2006, p. 1) 

Opportunity 

Given the literature on the criticality of self-understanding related to learning 
and motivation, and the evidence that Our Kids are unlikely to experience 
life situations that promote the development of these capacities, including 
these aspects in the definition of college readiness is an opportunity to 
recognize their importance to the success of Our Kids. In addition, inclusion 
might ensure that instruction that will further academic behaviors is included 
in a college readiness learning system. If not explicitly stated as a defined 
outcome, the development of metacognitive and self-system thinking, which 
are much harder to teach and harder to measure than content knowledge, 
may be given less attention in education systems under heavy content- 
specific assessment and accountability structures. 

Recommendations 

The findings from this report lead the researchers to one overarching 
recommendation with several tactical options for implementation. In order 
to capitalize on the strong work that has already been done in the area of 
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defining college readiness and to shore up those 
areas that are most critical for Our Kids, the 
Stupski Voundation should develop its own definition of 
college readiness for Our Kids that acknowledges the critical 
nature of content knowledge and also articulates the role 
of metacognition and development of the self-sjstem as 
components for successful preparation for college. 

To maximize this opportunity, the findings 
presented in this report and the options discussed 
should form the foundational criteria for a 
definition of college readiness for Our Kids. One 
or more of three approaches described below 
may be taken in defining college readiness, based 
on the impact on schools and districts. They are 
referred to as the Cafeteria, the Conversion, and the 
Composition approaches. 

Option 1: 

The Cafeteria approach 

The Cafeteria approach recognizes the significant 
and empirically-based efforts that have already 
been made in identifying rigorous, subject-area 
standards which include content associated with 
student success in post-secondary institutions. In 
addition, at least one program, the International 
Baccalaureate, has strong historical evidence 
to show that its graduates are prepared for 
post-secondary work, although this evidence 
is not specific to Our Kids. In the Cafeteria 
approach, districts engaged in implementing the 
Stupski Foundation’s Learning System would be 
permitted to adopt programs and standards from 
a “menu” of those the Foundation has determined 
have either shown promise or reflect a strong 
research base, and to which they have added 
any metacognitive and self-system components 
deemed essential for Our Kids. 

Potential benefit of this option 
The strength of such an approach is that it 
recognizes that districts, regardless of the 
similarities among their students, are situated in 


different political environments. Each state has its 
own standards and assessment and accountability 
requirements, which impacts a district’s choices. 
For example, the district may have little or no 
choice about the textbooks used. 

Potential drawback of this option 
The challenge of the Cafeteria approach is 
establishing a set of guidelines to ensure that 
adopted programs are viable and worthwhile 
in helping Our Kids succeed. This means 
examining aU potential programs or sets of 
available standards, and making sure the adopted 
curriculum includes those personal attributes of 
students that have been identified as critical. 

Option 2: 

The Conversion approach 

The Conversion approach means that aU Stupski 
partner districts would be directed to adopt, or 
convert to, the same approach. For example, aU 
districts might become IB schools, or aU districts 
might adopt the KSUS standards and a program 
that supports them. This approach assumes that 
the chosen program has been reviewed in-depth 
prior to its adoption; appropriate supplementation 
to ensure that Our Kids are well-served has been 
developed; and that it is supported by districts, 
schools, and stakeholders. 

Potential benefit of this option 

The advantage of this approach is consistency 
across the system and the associated economies of 
scale. Also, the districts may receive support from 
the program provider in terms of resources and 
in making the case to others — in the community 
as well as in higher education — that the program 
adequately prepares students for college. 

Potential drawback of this option 

The disadvantage is that the chosen approach might 
not adequately reflect the district’s unique context. 
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Option 3: 

The Composition approach 

In the Composition approach, college readiness is defined by Stupski for 
its partner districts, and the curriculum and program of instruction are 
custom-built or adapted from extant curricula to ensure that students will 
meet the requirements of the new definition. The name for this approach. 
Composition, reflects identifying and bringing together standards and 
programs from across organizations. The work begins by identifying the 
best of current thinking across aU academic subjects, synthesizing content 
specifics across standards, and identifying the strengths of each, with 
attention to the cognitive strategies that ensure student success. The work 
reviews the set of academic behaviors in order to establish a common 
language, making clear the metacognitive and self-system skills that 
characterize the successful student. 

The recent announcement from the U.S. Department of Education 
regarding proposed uses of the Race to the Top grant funds (Race to the 
Top Fund, 2009) provides a strong signal regarding the importance of the 
Common Core State Standards Initiative of the CCSSO and NGA, in that 
for states to be competitive for this funding they must indicate that they 
intend to adopt a common set of K-I2 standards. Given this, it might be 
possible to build the Composition approach upon the framework of national 
standards, and then add those metacognitive and self-system components 
essential to the success of Our Kids. 

Potential benefit of this option 

The advantage of this approach is that it is innovative yet capitalizes on work 
already done; that it realizes economies of scale by developing common 
goals across the Learning System; and it ensures that every curriculum used 
across every district and grade is designed to help prepare students for 
college. In this approach, any curriculum could be reviewed for its suitability 
in promoting aU aspects of the Stupski— established definition, although not 
every curriculum may be acceptable. 

Potential drawbacks of this option 

There are, of course, significant challenges to this approach. In the event 
that national standards are not a sufficient default foundation, developing 
an entirely new set of content standards requires considerable time, effort, 
and expense. In addition, the Foundation may need to accommodate variable 
accountability frameworks in place for the different states with which it 
works. There wiU be uncertainty about the program’s viability, as well, given 
its newness. Although there may be proxies to check the soundness of the 
approach, it wiU take time for a cohort of students to succeed in the first 
year of college and confirm the viabiUty of the definition. 
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The path forward 

Whatever approach or variant of an approach is 
taken — Cafeteria, Conversion, or Composition — 
it is necessary to determine the aspects of college 
readiness that must be present, including content 
knowledge, cognitive strategies, contextual skills, 
academic behaviors and the metacognitive and 
self-system thinking that underlie such behaviors. 
The approach taken will affect the level of work 
required to satisfactorily address the needs of 
districts and schools so that they can ensure that 
Our Kids are college ready. 

At one end of the continuum, this work may 
mean that the definition of college readiness 
entails specifying guidelines to ensure that districts 
and schools have adopted essential components 
(standards and programs) that will meet specified 
criteria for each of the four categories of 


content knowledge, cognitive strategies, academic 
behaviors, and contextual skills. In such a case, 
the operational definition is the establishment 
of criteria for evaluating whether each category 
is fuUy addressed. At the other end of the 
continuum, the work is to identify and specify 
content, the set of cognitive strategies, academic 
behaviors, and related metacognitive and self- 
system thinking expected of each student, as well 
as the extent and depth of contextual skills, and 
the curriculum and supports needed to implement 
the program. 

Regardless of the approach, the Foundation 
must identify the critical elements from the 
literature, and from this report, that it deems 
most important to include in its definition and 
then ensure that those elements appear in the 
definition developed. 
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Final Thoughts 


Defining college readiness for aU students in the 
2D' century in America today is a formidable task. 
This report identified strong areas of agreement 
in the literature — the importance of content 
knowledge and metacognitive and self-system 
thinking — but also highlighted a dilemma for the 
Foundation in attempting to identify an existing 
definition upon which to rely. No single definition 
appears to provide aU of the essential ingredients 
for Our Kids. 


A satisfactory definition of college readiness 
for Our Kids can only be informed by research. 
Ultimately, the Learning System and Our Kids will 
best be served by a definition that is generated 
from the same spirit and fervor upon which 
the entire mission of the Foundation is based. 

It win be, after all, the target toward which the 
“campaign” will point. It will frame the work of 
those currently involved, those yet to join, and the 
lives of thousands who depend upon the success 
of the effort. 


Stupski Foundation's Learning System / College Readiness 25 


26 


References 


Achieve, Inc. (2004). The expectations gap: A 50-state review of high school graduation requirements. 
Washington, DC: Author. 

Achieve, Inc. (2005). Rising to the challenge: Are high school graduates prepared for college and 
M/or^f.? Washington, DC: Author. 

Achieve, Inc. (2008). The building blocks of success: Higher-level mathematics for all students. 
Washington, DC: Author. 

ACT. (2005). Courses count: Preparing students for postsecondary success. Iowa City, lA: Author. 

ACT. (2006). Ready for college and ready for work: Same or different? Iowa City, lA: Author. 

Adelman, C. (1999). Answers in the tool box: Academic intensity, attendance patterns, and bachelor's 
degree attainment. Washington, DC: U.S. Department of Education, Office of Educational 
Research and Improvement. 

Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through 
college. Washington, DC: U.S. Department of Education, Office of Vocational and Adult Education. 

Allen, J., & Sconing, J. (2005). Using ACT assessment scores to set benchmarks for college readiness. 
Iowa City, lA: ACT. 

American Council on Education. (2008). Minorities in higher education 2008 twenty-third status 
report. Washington, DC: Author. 

American Diploma Project. (2004). Ready or not? Creating a high school diploma that counts. 
Washington, DC: Achieve, Inc. 

Barton, P. E., & Lapointe, A. (1995). Learning by degrees: Indicators of performance in higher 
education. Princeton, NJ: Educational Testing Service. 

Beers, K. (2009). The genteel unteaching of America's poor. Urbana, IL: National Council of Teachers 
of English. Retrieved from http://www.ncte.org. 

Conley, D. T. (2003a). Understanding university success: A report from Standards for Success. 
Retrieved from http://www.s4s.org/03_viewproducts/ksus/pdf/Understanding_Success.pdf. 

Conley, D. T. (2003b, April 22). What is the alignment between knowledge and skill for university 
success and state academic content standards? Paper presented at the annual meeting of the 
American Educational Research Association, Chicago, IL. 

Conley, D. T. (2007). Toward a more comprehensive conception of college readiness. Eugene, OR: 
Educational Policy Improvement Center. 

Design Collaborative, (n.d.). Retrieved from http://www.stupski.org/design_collaborative.htm 


Stupski Foundation's Learning System / College Readiness 27 


Duncan, G. J. & Magnuson, K. A. (2005). Can family socioeconomic resources account for racial and 
ethnic test score gaps? The Future of Children, 75(1), 35-54. 

Friedman, T. L. (2006). The world is at: A brief history of the twenty-first century. (2"'' ed.). New York: 
Farrar, Straus and Giroux. 

Goldberger, S., & Bayerl, K. (2008). Beating the odds: The real challenges behind the mathematics 
achievement gap— and what high-achieving schools can teach us about how to close it. Boston: 
Jobs for the Future. 

Haycock, K., & Huang, S. (2001, Winter). Are today's high school graduates ready? Thinking K-16, 

5(1), 3-22. 

Hill, I. (2008, January/February). Bac to the future: The International Baccalaureate. Edge, 5(3), 1-20. 

Hoff, D. J. (2009). National standards gain steam: Governors' embrace rooted in competitiveness 
concern. Education Week, 28(23), 1-20. 

International Baccalaureate Organization. (2006). IB learner profile booklet. Cardiff, Wales: Author. 

Kendall, J. S., Pollack, C., Schwols, A., & Snyder, C. (2007). High school standards and expectations 
for college and the workplace. (Issues & Answers Report, REL 2007-No. 001 ). Washington, DC: 

U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Evaluation and Regional Assistance, Regional Educational Laboratory Central. 

Kendall, J. S., Ryan, S., Weeks, S., Alpert, A., Schwols, A., & Moore, L. (2008). Thinking & iearning 
skills: What do we expect of students? Denver, CO: Mid-continent Research for Education and 
Learning. 

Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2006). What matters to student 
success: A review of the iiterature. Washington, DC: National Postsecondary Education 
Cooperative. 

Levine, M. (2007, April). The essential cognitive backpack. Educational Leadership, 64(7), 16-22. 

Martinez, M., & Klopott, S. (2005). The link between high school reform and college access and 
success for low-income and minority youth. Washington, DC: American Youth Policy Forum and 
Pathways to College Network. 

Marzano, R. J. & Kendall, J. S. (2007). The new taxonomy of educationai objectives. (2'"‘ ed.) 

Thousand Oaks, CA: Corwin Press. 

North Central Regional Educational Laboratory. (2003). enGauge: 21^ century skiils for 2T' century 
learners. Naperville, IL: Author. 

Olson, L. (2006). Views differ on defining college prep. Education Week, 25(33), 1-29. 

Organisation for Economic Co-operation and Development. (2005). The definition and selection of key 
competencies: Executive Summary. Paris: Author. 

Partnership for 21®' Century Skills. (2006). Resuits that matter: 27®' century skills and high school 
reform. Tucson, AZ: Author. 


28 


Pink, D. H. (2005). A whole new mind: Why right-brainers will rule the future. New York: Riverhead 
Books. 

Race to the Top Fund, RIN 1810-AB07, 55 Fed. Reg. 74, 144 (July 29, 2009). 

Robbins, S. B., Lauver, K., Le, FI., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and 
study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130(2), 
261-288. 

Robbins, S. B., Allen, J., Casillas, A., Peterson, C. FI., & Le, FI. (2006). Unraveling the differential effects 
of motivational and skills, social, and self-management measures from traditional predictors of 
college outcomes. Journal of Educational Psychology, 98(3), 598-616. 

Stupski Foundation, (n.d.). R & D for innovation in public education. Retrieved from http://www. 
stupski.org/rdforinnovation.htm. 

Stupski Foundation. (2008). Strategy and program overview. San Francisco: Author. 

The College Board. (2006a). College Board standards for college success: English language arts. New 
York: Author. 

The College Board. (2006b). College Board standards for college success: Mathematics and statistics. 
New York: Author. 


Stupski Foundation's Learning System / College Readiness 29 


30 


References Reviewed but Not Cited in This Report 


Achieve, Inc. (2004). Do graduation tests measure up? A closer look at state high school exit exams. 
Washington, DC: Author. 

Achieve, Inc. (2006). Closing the expectations gap 2006. Washington, DC: Author. 

Achieve, Inc. (2006). Identifying potential dropouts: Key lessons for building an early warning data 
system. Washington, DC: Author. 

Achieve, Inc. (2007). Aligned expectations? A closer look at college admissions and placement tests. 
Washington, DC: Author. 

Achieve, Inc. (2007). Aligning high school graduation requirements with real world: A road map for 
states. Washington, DC: Author. 

Achieve, Inc. (2007). Closing the expectations gap 2007. Washington, DC: Author. 

Achieve, Inc. (2007). Creating a world-class education system in Ohio. Washington, DC: Author. 
Achieve, Inc. (2008). Algebra II end-of-course exam: 2008 annual report. Washington, DC: Author. 
Achieve, Inc. (2008). Closing the expectations gap. Washington, DC: Author. 

Achieve, Inc. (2008). Out of many, one: Toward rigorous common core standards from the ground up. 
Washington, DC: Author. 

Achieve, Inc. (2008). R/s/c reversa/.? Washington, DC: Author. 

Achieve, Inc. (2008). The Perkins Act of 2006: Connecting career and technical education with the 
college and career readiness agenda. Washington, DC: Author. 

Achieve, Inc. (n.d.). Creating a high school diploma that counts. Washington, DC: Author. 

Achieve, Inc. (n.d.). Preparing today's high school students for tomorrow's opportunities. 

Washington, DC: Author. 

Achieve, Inc., and National Governors Association. (2005). An action agenda for improving America's 
high schools. Washington, DC: Author. 

ACT. (2005). Crisis at the core: Preparing all students for college and work. Iowa City, lA: Author. 

ACT. (2005). On course for success: A close look at selected high school courses that prepare all 
students for college. Iowa City, lA: Author. 

ACT. (2006). Developing the STEM education pipeline. Iowa City, lA: Author. 

ACT. (2006). Reading between the lines: What the ACT reveals about college readiness in reading. 
Iowa City, lA: Author. 


Stupski Foundation's Learning System / College Readiness 31 


ACT. (2006). Ready to succeed: All students prepared for college and work. Iowa City, lA: Author. 

ACT. (2007). ACT national curriculum survey 2005-2006. Iowa City, lA: Author. 

ACT. (2007). Aligning postsecondary expectations and high school practice: The gap defined. Policy 
implications of the ACT national curriculum survey results 2005-2006. Iowa City, lA: Author. 

ACT. (2007). Family firsts: Reach for your dreams. Iowa City, lA: Author. 

ACT. (2007). Rigor at risk: Reaffirming quality in the high school core curriculum. Iowa City, lA: 

Author. 

ACT. (2007). State of college readiness for Latino students. Iowa City, lA: Author. 

ACT. (2007). The role of nonacademic factors in college readiness and success. Iowa City, lA: Author. 

ACT. (2007). Using EXPLORE and PLAN data to evaluate GEAR UP programs. Iowa City, lA: Author. 

ACT. (2008). Making the dream a reality: Action steps for states to prepare all students for college and 
career. Iowa City, lA: Author. 

ACT. (2008). The economic benefits of academic and career preparation. Iowa City, lA: Author. 

ACT. (2008). The forgotten middle: Ensuring that all students are on target for college and career 
readiness before high school. Iowa City, lA: Author. 

ACT. (2008). What we know about college success: Using ACT data to inform educational issues. 

Iowa City, lA: Author. 

Ad Hoc Committee on Planning for NAEP 12’*' Grade Assessments in 2009. (2006). The future of 12‘^ 
grade NAEP: Report of the Ad Hoc Committee on Planning for NAEP 12'^ Grade Assessments in 
2009. Washington, DC: National Assessment Governing Board. 

Adelman, C. (2004). Principal indicators of student academic histories in postsecondary education, 
1972-2000. Washington, DC: U.S. Department of Education, Institute of Education Sciences. 

Allensworth, E. M., & Nomi, T. (2009). College-preparatory curriculum for all: The consequences 
of raising mathematics graduation requirements on students' course taking and outcomes in 
Chicago. Chicago: Consortium on Chicago School Research at the University of Chicago. 

Allensworth, E., Correa, M., & Ponisciak, S. (2008). From high school to the future: ACT preparation — 
too much, too late. Chicago: Consortium on Chicago School Research at the University of 
Chicago. 

Alvarez, D., & Mehan, H. (n.d.). Whole school detracking: A strategy for equity and excellence. Theory 
into Practice, 45(1), 82-89. 

Arkansas Department of Education. (2007). Combined research report of business leaders and college 
professors on preparedness of high school graduates. Little Rock, AR: Author. 

Atkinson, R. C., & Pelfrey, P. A. (2004). Rethinking admissions: U.S. public universities in the post- 
Affirmative Action age. Berkeley, CA: University of California Center for Studies in Higher 
Education. 


32 


Baird, L. L. (1983). Predicting predictability: The in uence of student and institutional characteristics 
on the prediction of grades. New York: Educational Testing Service. 

Balloon, E. G. (2008). Racial differences in high school math track assignment. Journal of Latinos and 
Education, 7(4), 272-287. 

Bangser, M. (2008). Evaluating the impact of interventions that promote successful transitions from 
high school. Washington, DC: National High School Center. 

Bangser, M. (2008). Preparing high school students for successful transitions to postsecondary 
education and employment. Washington, DC: National High School Center. 

Barth, P. (2003, Winter). A common core curriculum for the new century. Thinking K-16, 7(1 ), 3-31 . 

Barton, P. E. (2005). One-third of a nation: Rising dropout rates and declining opportunities. 

Princeton, NJ: Educational Testing Service. 

Barton, P. E. (2006). High school reform and work: Facing labor market realities. Princeton, NJ: 
Educational Testing Service. 

Bayerl, K. (2007). Rigor plus support: How science teachers use literacy techniques to get students 
ready for college. Boston: Jobs for the Future. 

Bill and Melinda Gates Foundation. (2003). Closing the graduation gap: Toward high schools that 
prepare all students for college, work, and citizenship. Seattle, WA: Author. 

Biswas, R. R. (2005). Our success is our graduates: Case study of Year Up: A career advancement 
model for low-income young adults. Boston: Jobs for the Future. 

Bloom, J. L. (2007). (Mis)reading social class in the journey towards college: Youth development in 
urban America. Teachers College Record, 709(2), 343-368. 

Borman, G. D., Stringfield, S. & Rachuba, L. (2000). Advancing minority high achievement: National 
trends and promising programs and practices. New York: The College Board. 

Boser, U., & Burd, S. (2009). Bridging the gap: How to strengthen the Pk-16 pipeline to improve 
college readiness. Washington, DC: New America Foundation. 

Bottoms, G. & Young, M. (2008). Lost in transition: Building a better path from school to college and 
careers. Atlanta, GA: Southern Regional Education Board. 

Bozick, R., & Ingels, S. J. (2008). Mathematics coursetaking and achievement at the end of high 

school: Evidence from the Education Longitudinal Study of 2002. Washington, DC: National Center 
for Education Statistics, Institute of Education Sciences, U.S. Department of Education. 

Cabrera, A. F., Deil-Amen, R., Prabhu, R., Terenzini, P. T., Lee, C., & Franklin, Jr., R. E. (2006). 

Increasing the college preparedness of at-risk students. Journal of Latinos and Education, 5(2), 
79-97. 


Callan, P. M., Finney, J. E., Kirst, M. W., Usdan, M. D., & Venezia, A. (2006). Claiming common 
ground: State policymaking for improving college readiness and success. San Jose, CA: The 
National Center for Public Policy and Higher Education. 


Stupski Foundation's Learning System / College Readiness 33 


Cannon, J. S., & Lipscomb, S. (2008). Changing the kindergarten cutoff date: Effects on California 
students and schools. San Francisco, CA: Public Policy Institute of California. 

Carnevale, A. P. & Desrochers, D. M. (n.d.). Connecting education standards and employment: Course- 
taking patterns of young workers. Washington, DC: Achieve, Inc., American Diploma Project. 

Cavanagh, S. (2004). Barriers to college: Lack of preparation vs. financial need. Education Week, 
23(19), 1-13. 

Cech, S. J., & Holovach, R. (2008). AP trends: Tests soar, scores slip. Education Week, 27(24), 1-13. 

Center for an Urban Future. (2006). Ready or not. New York: Author. 

Center for Fligher Education Policy Analysis. (2002). Making the grade in college prep: A guide 
for improving college preparation programs. Los Angeles: Center for Fligher Education Policy 
Analysis, University of Southern California. 

Center for the Study of Education Policy. (2005). Improving chances for college success for low 
income and minority high school students. Normal, IL: Author. 

Committee for Economic Development. (2005). Cracks in the education pipeline: A business leader's 
guide to higher education reform. Washington, DC: Author. 

Conley, D. (2003). Mixed messages: What state high school tests communicate about student 
readiness for college. Eugene, OR: Center for Education Policy Research, University of Oregon. 

Conley, D. T. (2003). Connecting the dots: Linking high schools and postsecondary education to 
increase student success. PeerReview, 5(2], 9-12. 

Conley, D. T. (2005). College knowledge: Getting in is only half the battle. Principal Leadership, 6(1), 
16-21. 

Conley, D. T. (2006). What we must do to create a system that prepares students for college success. 
Policy Perspectives. San Francisco: WestEd. 

Conley, D. T. (2007, April). The challenge of college readiness. Educational Leadership, 64(7), 23-29. 

Conley, D. T. (2008, Spring). Rethinking college readiness. The New England Journal of Higher 
Education, 22(5), 24-26. 

Conley, D. T., & Venezia, A. (2003). High school transitions: State of the art and views of the future. 
Paper presented at the Annual Meeting of the American Educational Research Association, 
Chicago, IL. 

Consortium for Policy Research in Education. (2000). Bridging the K-1 2/postsecondary divide with a 
coherent K-16 system. Philadelphia: Author. 

Corrallo, S. (1996). The National Assessment of College Student Learning: An inventory of state-level 
assessment activities. Washington, DC: National Center for Education Statistics. 

Crist, C., Jacquart, M., & Shupe, D. A. (2002). Improving the performance of high school students: 
Focusing on connections and transitions taking place in Minnesota. Washington, DC: Office of 
Vocational and Adult Education. 


34 


Darling-Hammond, L., & Friedlaender, D. (2008). Creating excellent and equitable schools. 

Educational Leadership, 65(8), 14-21. 

Dougherty, C., & Zavadsky, H. (2007, November). Giving all students the keys to college and skilled 
careers: One district's approach. Phi Delta Kappan, 89(3), 194-199. 

Dounay, J. (2006). Ensuring rigor in the high schooi curriculum: What states are doing. Denver, CO: 
Education Commission of the States. 

Editorial Projects in Education Research Center. (2008). School to college: Can state P-16 councils 
ease the transition? Retrieved January 15, 2009, from www.edweek.org/rc 

Education Commission of the States. (2005, May). Dual enrollment. The progress of education reform 
2005, 6(3). 

Epstein, K. K. (2004, June). Small schools— Miracle school: A child of the civil rights movement. Phi 
Delta Kappan, 85(10), 773-776. 

Ewell, P. & Wellman, J. (2007). Enhancing student success in education: Summary Report of the 
NPEC Initiative and Nationai Symposium on Postsecondary Student Success. Washington, DC: 
National Postsecondary Education Cooperative. 

Fadigan, K. A., & Hammrich, P. L. (2004). A longitudinal study of the educational and career 

trajectories of female participants of an urban informal science education program. Journal of 
Research In Science Teaching, 41(8), 835-860. 

Farmer-Flinton, R. L. (2008). Creating opportunities for college access: Examining a school model 
designed to prepare students of color for college. Multicultural Perspectives, 10(2), 73-81. 

Field, K. (2008). Education Department to end controversial study of Upward Bound. Chronicle of 
Higher Education, 54(26), A20-A23. 

Finkelstein, N. D., & Fong, A. B. (2008). Course taking patterns and preparation for postsecondary 
education in California's public university systems among minority youth. Washington, DC: 

U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Evaluation and Regional Assistance, Regional Education Laboratory West. 

Gandara, P. (2005). Fragile futures: Risk and vuinerability among Latino high achievers. Princeton, NJ: 
Educational Testing Service. 

George, P., & Aronson, R. (2003). How do educators' cultural belief systems affect underserved 
students' pursuit of postsecondary education? Hono\u\u, Fll: Pacific Resources for Education and 
Learning (PREL). 

Gewertz, C. (2009, January 13). Redouble high school improvement push, states told. Education 

Week. Retrieved January 14, from http://www.edweek.org/ew/articles/2009/01/13/18readiness.h28. 
html 

Glassman, P., & Roelle, R. J. (2007, November). Singling black boys to close the gaps. School 
Administrator, 64(10), 26-31. 


Stupski Foundation's Learning System / College Readiness 35 


Gomoran, A., Porter, A. C., Smithson, J., & White, 0. A. (1997). Upgrading high school mathematics 
instruction: Improving learning opportunities for low-achieving, low-income youth. Educational 
Evaluation and Policy Analysis, 79(4), 325-338. 

Gong, Y., & Presley, J. B. (2006). The demographics and academics of college going in Illinois. 
Edwardsville, IL: Illinois Education Research Council. 

Greene, J. P., & Winters, M. A. (2003). Public high school graduation and college readiness rates: 
1991-2002. New York: Center for Civic Innovation at the Manhattan Institute. 

Greene, J. P., & Winters, M. A. (2005). Public high school graduation and college readiness rates: 
1991-2002. New York: Center for Civic Innovation at the Manhattan Institute. 

Greenwood, A. (1994). The National Assessment of College Student Learning: Identification of skills 
to be taught, learned, and assessed. Washington, DC: National Center for Education Statistics. 

Hauptman, A. M. (2007). Strategies for improving student success in postsecondary education. 
Boulder, CO: Western Interstate Commission for Higher Education. 

Haycock, K., & Gerald, D. (2007). Trend: Shrinking opportunity. Connection: The Journal of the New 
England Board of Higher Education, 21(5), 15-16. 

Heisel, M. (2005). Preparing New Jersey students for college and careers: Evaluation of New Jersey 
GEAR UP. Trenton, NJ: New Jersey Commission on Higher Education. 

Hoachlander, G. (2008, May). Bringing industry to the classroom. Educational Leadership, 65(8), 
22-27. 


Howard-Vital, M. (n.d.). Untangling our web: A statewide approach is needed to improve/judge 
college readiness and to increase college success. Chapel Hill, NC: University of North Carolina. 

Huber, L. P., Huidor, 0., Malagon, M. C., Sanchez, G., & Solorzano, D. G. (2006). Falling through the 
cracks: Critical transitions in the Latino/a educational pipeline. Los Angeles: UCLA Chicane Studies 
Research Center. 

Huebner, T. A. (2007). Rethinking high school: Preparing students for success in college, career, and 
life. Seattle, WA: Bill and Melinda Gates Foundation. 

Huebner, T. A., & Corbett, G. C. (2004). Rethinking high school: Five profiles of innovative models for 
student success. San Francisco: WestEd. 

Karp, M. M., & Hughes, K. L. (2008, April). Supporting college transitions through collaborative 
programming: A conceptual model for guiding policy. Teachers College Record, 110(A), 838-866. 

Karp, M. M., Calcagno, J. C., Hughes, K. L., Jeong, D. W., & Bailey, T. R. (2007). The postsecondary 
achievement of participants in dual enrollment: An analysis of student outcomes in two states. St. 
Paul, MN: National Research Center for Career and Technical Education. 

Kendall, J. S., & Snyder, C. (2005). Challenging standards in English: Are high schools missing 
essential content? Denver, CO: Mid-continent Research for Education and Learning. 

Kendall, J. S., & Williams, J. (2004). Challenging standards in mathematics: What are high schools 
missing? Denver, CO: Mid-continent Research for Education and Learning. 


36 


Kentucky Developmental Education Task Force. (2007). Securing Kentucky's future: A plan for 
improving college readiness and success. Frankfort, KY: Author. 

Kirst, M., & Venezia, A. (2001 , September). Bridging the great divide between secondary schools and 
postsecondary education. Phi Delta Kappan, 83(^), 92-97. 

Lopez, D. (2009, January/February). College readiness for all: What's the alternative? Principal, 50-51. 

Lundell, D. B., Higbee, J. L., Duranczyk, I. M., & Goff, E. (Eds.) (n.d.). Student standpoints about 
access programs in higher education. Minneapolis, MN: Center for Research on Developmental 
Education and Urban Literacy. 

Manzo, K. K., & Cavanagh, S. (2008, April 23). America scouts overseas to boost education skills. 
Education Week, 27(34), 14-16. 

Mayer, A. P. (2008, Winter). Expanding opportunities for high academic achievement: An 

International Baccalaureate diploma program in an urban high school. Journal of Advanced 
Academics, 79(2), 202-235. 

McLendon, M., Heller, D. E., & Lee, S. (2009). High school to college transition policy in the American 
states: Conceptual and analytic perspectives on conducting across-state study. Educational Policy, 
23(2), 385-418. 

McNeil, M. (2008, March 12). Benchmarks momentum on increase. Education Week, 27(27), 1, 12-13. 

Meeder, H., & Solares, G. (2006). Perspectives on English and math skills needed for work readiness. 
Idel, lA: Visions Unlimited, Inc. 

Minnesota Higher Education Coordinating Board. (1986). Defining college-level skills. St. Paul, MN: 
Author. 

Muraskin, L. (2003). National evaluation of GEAR UP: A summary of the first two years. Rockville, MD: 
Westat, Inc. 

National Association of System Heads. (2002). Aligning K-12 and postsecondary expectations: State 
policy in transition. Washington, DC: Author. 

National Center for Higher Education Management Systems. (1994). A preliminary study of 
the feasibility and utility for national policy of instructional "good practice" indicators in 
undergraduate education. Washington, DC: National Center for Education Statistics. 

National Commission on NAEP 12*'" Grade Assessment and Reporting. (2004). 72"' grade student 
achievement in America: A new vision for NAEP. Washington, DC: Author. 

National Governors Association, Council of Chief State School Officers, & Achieve, Inc. (2008). 
Benchmarking for success: Ensuring U.S. students receive a world-class education. Washington, 
DC: National Governors Association. 

National Governors Association. (2003). Ready for tomorrow: Helping all students achieve secondary 
and postsecondary success. Washington, DC: Author. 

Nevarez, C., & Rico, T. (2007). Latino education: A synthesis of recurring recommendations and 
solutions in P-16 education. New York: The College Board. 


Stupski Foundation's Learning System / College Readiness 37 


Noble, J. P., & Sawyer, R. L. (2004). Is high school GPA better than admission test scores for 
predicting academic success in college? College and University (79)4, 17-22. 

Noeth, R. J., & Wimberly, G. L. (2002). Creating seamless educational transitions for urban African 
American and Hispanic students. Iowa City, I A: ACT, Inc. 

Olson, L. (2006). States push to align policies from pre-K to postsecondary. Education Week, 25(41), 
1 - 20 . 


Oswald, K. J. (2002). The AVID program in AISD, 1999-2002. Austin, TX: Austin Independent School 
District, Office of Program Evaluation. 

Parsad, B., Lewis, L., & Greene, B. (2003). Remedial education at degree-granting postsecondary 
institutions in Fall 2000. Washington, DC: National Center for Education Statistics. 

Pathways to College Network, Institute for Higher Education Policy, & Social Science Research 
Council. (2007). Questions that matter: Connecting research, policy, and practice to improve 
college access and success. Boston: Authors. 

Perna, L. W., & Thomas, S. L. (2006). A framework for reducing the college success gap and 
promoting success for all. Washington, DC: National Postsecondary Education Cooperative. 

Plenty, M., Provasnik, S., & Daniel, B. (2007). High school coursetaking: Findings from the Condition 
of Education 2007. Washington, DC: U.S. Department of Education, Institute of Education 
Sciences. 

Quint, J., Thompson, S. L., & Bald, M. (2008). Relationships, rigor, and readiness: Strategies for 
improving high schools. New York: MDRC. 

Ramist, L., Lewis, C., & McCamley-Jenkins, L. (1994). Student group differences in predicting college 
grades: Sex, language, and ethnic groups. New York: College Entrance Examination Board. 

Richburg-Hayes, L. (2008). Helping low-wage workers persist in education programs. New York: 
MDRC. 

Roderick, M. (2006). Closing the aspirations-attainment gap: Implications for high school reform. New 
York: MDRC. 

Roderick, M., Nagoaka, J., Coca, V., Moeller, E., Roddie, K., Gilliam, J., & Patton, D. (2008). From high 
school to the future: Potholes on the road to college. Chicago: Consortium on Chicago School 
Research at the University of Chicago. 

Sawyer, R. (2008). Benefits of additional high school course work and improved course performance 
in preparing students for college. Iowa City, lA: ACT. 

Schmidt, P. (2008). New twists mark the debate over Texas' top 10-percent plan. Chronicle of Higher 
Education, 54(39), A20-A24. 

Seftor, N. S., Mamun, A., & Schirm, A. (2008). The impacts of regular Upward Bound on 

postsecondary outcomes 7-9 years after scheduled high school graduation. Princeton, NJ: 
Mathematica Policy Research, Inc. 


38 


Silva, E. (2008). Measuring skills for the 2P‘ century. Washington, DC: Education Sector. 

Smith, A. (n.d.). Equity within reach: Insights from the front lines of America's achievement gap. New 
York: Teach for America. 

Snipes, J. C., Holton, G. I., Doolittle, F., & Sztejnberg, L. (2006). Striving for student success: The 
effect of Project GRAD on high school student outcomes in three urban school districts. New York: 
MDRC. 

Society for Human Resource Management. (2007). SHRM 2007 symposium on the workforce 
readiness of the future U.S. labor pool. Washington, DC: Author. 

Southern Regional Education Board. (2005). Creating college opportunity for all: Prepared students 
and affordable colleges. Atlanta, GA: Author. 

Southern Regional Education Board. (2006). Getting students ready for college and careers. Atlanta, 
GA: Author. 

Southern Regional Education Board. (2007). High school to college and careers: Aligning state 
policies. Atlanta, GA: Author. 

Sparks, S. D. (2007, May 30). International group faults U.S. schools. Education Daily, 40(101), 1-4. 

St. Jarre, K. (2008, October). They knew calculus when they left: The thinking disconnect between 
high school and university. Phi Delta Kappan, 90(2),123-126. 

Standing, K., Judkins, D., Keller, B., & Shimshak, A. (2008). Early outcomes of the GEAR UP program. 
Rockville, MD: Westat. 

State Higher Education Executive Officers. (2003). Student success: Statewide P-16 systems. Denver, 
CO: Author. 

State University of New York. (1995). College expectations: Knowledge & skills. Albany, NY: Author. 

Stein, M., Robinson, S., Haycock, K., Vitale, D., & Schmeiser, C. (2005, September). College Prep 101. 
Principal Leadership, 6(1), 22. 

Stringer, N. (2008). Aptitude tests versus school exams as selection tools for higher education and 
the case for assessing educational achievement in context. Research papers in education, 23(1), 
53-68. 


Strong American Schools. (2008). Diploma to nowhere. Washington, DC: Author. 

Terenzini, P. T., Cabrera, A. F., & Bernal, E. M. (2001 ). Swimming against the tide: The poor in 
American higher education. New York: The College Board. 

The College Board. (2007). The college keys compact: Getting ready, getting in, and getting through 
college: Expanding options for low-income students. New York: Author. 

The College Board. (2008). Resources for increasing Latino participation and success in higher 
education. New York: Author. 


Stupski Foundation's Learning System / College Readiness 39 


The Conference Board, the Partnership for 21®* Century Skills, Corporate Voices for Working 

Families, & the Society for Fluman Resource Management. (2006). Are they really ready to work? 
Employers' perspectives on the basic knowledge and applied skills of new entrants to the 27®' 
century U.S. workforce. New York: Author. 

The Education Trust-West. (2004). Are California high schools ready for the 2P‘ century? Oakland, CA: 
Author. 

The New Commission on the Skills of the American Workforce. (2008). Tough choices or tough times. 
Washington, DC: National Center on Education and the Economy. 

The Secretary's Commission on Achieving Necessary Skills. (1991 ). What work requires of schools: a 
SCANS report for America 2000. Washington, DC: U.S. Department of Labor. 

Tierney, W. G., & Jun, A. (2001 ). A university helps prepare low income youths for college. The 
Journal of Higher Education, 72(2), 205-225. 

Tinto, V. (2008). When access is not enough. Stanford, CA: The Carnegie Foundation for the 
Advancement of Teaching. 

Trent, W. T., Orr, M. T., Ranis, S. H., & Holdaway, J. (2007, October). Transitions to college: Lessons 
from the disciplines. Teachers Coilege Record, 709(10), 2207-2221. 

Turner, P. R. (2008). A predictor-corrector process with refinement for first-year calculus transition 
support. Primus, 7S(4), 370-393. 

U.S. Department of Education, Office of Vocational and Adult Education. (2003). The high school 
leadership summit: Issue papers. Washington, DC: Author. 

Uy, E. (2008, September 9). Education groups spearhead international benchmarking. Education 
Daily, 47(165), 1-3. 

Van Noy, M., Jacobs, J., Korey, S., Bailey, T., & Hughes, K. L. (2008, Apr/May). The rise of noncredit 
workforce education. Community College Journal, 7S(5), 54-59. 

Venezia, A. (2003, Winter). Connecting the systems: What can postsecondary education do to work 
with K-12 to help students better prepare for college? PeerReview, 5(2), 27-30. 

Venezia, A. (2006, Winter). Levers for change: Steps states can take to improve college readiness. 
Connection: The Journal of the New England Board of Higher Education, 20(4), 16-18. 

Venezia, A., Callan, P. M., Finney, J. E., Kirst, M. W., & Usdan, M. D. (2005). The governance divide: 

A report on a four-state study on improving college readiness and success. Washington, DC: The 
National Center for Public Policy and Higher Education. 

Venezia, A., Kirst, M. W., Antonio, A. L. (n.d.). Betraying the coilege dream: How disconnected K-12 
and postsecondary education systems undermine student aspirations. Stanford, CA: The Stanford 
Institute for Higher Education Research. 

W. K. Kellogg Foundation, (n.d.). ENLACE: Creating college-focused centers in the schools: Learning 
from the ENLACE experience. Battle Creek, Ml: Author. 


40 


Wagner, A. (2006). Measuring up internationally: Developing skills and knowledge for the global 
knowledge economy. San Jose, CA: The National Center for Public Policy and Higher Education. 

Ward, N. L. (2006). Improving equity and access for low-income and minority youth into institutions 
of higher education. Urban Education, 47(1), 50-70. 

Warner, C. B., & Phelps, R. E. (n.d.). The relationship between motivational orientation & educational 
aspirations in urban, African American youth. Middle Grades Research Journal, 3(2), 71-85. 

Washington Higher Education Coordinating Board. (2008). Passport to college: Promise Scholarship 
Program status report. Olympia, WA: Author. 

Weissberg, N. C., & Owen, D. R. (2005). Do psychosocial and study skill factors predict college 
outcomes? Comment on Robbins et al. (2004). Psychological Bulletin 737(3), 407-409. 

White Ho use Initiative on Educational Excellence for Hispanic Americans. (2008). A summary of 
activities, accomplishments and Hispanic education attainment: 2001-08. Washington, DC: 
Author. 

Williams, N. J., & Noble, J. P. (2005). School-level benefits of using PLAN overtime. Iowa City, lA: 
ACT. 

Wilson, B. J. (2006, Summer). Preparing urban scholars for college: A best practice in college 
readiness. Connection:The New Journal of the New England Board of Higher Education. 37(1), 
15-16. 

Wimberly, G. L., & Noeth, R. J. (2005). College readiness begins in middle school. Iowa City, lA: ACT. 

Wright, D., & Bogotch, I. (2006, Fall). High school: Erasing borders. Journal of College Admission, 
193, 18-24. 


Stupski Foundation's Learning System / College Readiness 41 


42 


Appendix 


Literature review method 

In June 2008, the Stupski Foundation created a conceptual framework for the reinvention of 
American education. The framework identified seven essential components and focused on 
delivering 21=* century college readiness for all students, but especially for "Our Kids," children of 
color and poverty. The Foundation explained that "graduating all students from high school with the 
knowledge and skills that qualify them as 'college ready' is the most meaningful and measurable way 
to increase life choices and options for all children, but most especially children of color and poverty" 
(About the Foundation, para. 3). 

The Learning System includes four core teaching and learning components: Curriculum, 
Assessments, Pedagogy, and Supports. Surrounding these components, are three organizational 
components necessary to support the core: Leadership/Human Capital, Systems Diagnostics, and a 
Dashboard of College Readiness Indicators (College Readiness Learning System, n.d.). 

The Foundation envisions convening a Design Collaborative, a cross-sector group of researchers, 
practitioners, and designers from inside and outside education, to "define, develop and continually 
improve" (Design Collaborative, n.d.) all of the components. To orient Design Collaborative members 
to the accumulated and maturing knowledge base related to each of the components and to children 
of color and poverty, the Foundation contracted with Mid-continent Research for Education and 
Learning (McREL). McREL conducted eight literature reviews— one on each of the components 
plus one on Our Kids— to identify and integrate theories and philosophical perspectives, issues, 
scientifically based research practices, unmet needs, and innovations relevant to designing one or 
more of the system components to accelerate learning for Our Kids. 

This Appendix contains a description of the review method, including a general explanation of 
McREL's approach and descriptions of the particular procedures used for each phase of the review: 
identification of key hypotheses and research questions, literature search, identification and 
cataloguing of finds, and generating and communicating recommendations. 

McREL's overall approach 

Since the primary users of the reviews are the members of the Design Collaborative, the qualitative, 
iterative approach taken for the literature reviews sought to achieve the multiple goals of identifying 
emerging ideas, counterproductive orthodoxies, and promising practices relevant to the reinvention 
of the Learning System. Thus, eight research teams were assembled, each with one or more 
researchers familiar with the respective topic areas. 

Qualitative approach. A qualitative approach shares several practices with those of systematic 
reviews, including comprehensive searches and transparency to reduce bias, but it differs with 
respect to inclusion/exclusion criteria. Systematic reviews emphasize explicit and a priori inclusion/ 
exclusion criteria and criteria for evaluating the methodological quality of individual studies, carefully 
limiting the sources of evidence to support inferences about cause and effect relationships (Cooper, 
Fledges, & Valentine, 2009). The qualitative approach emphasizes diverse sources and types of 
evidence and knowledge to support a broader base of inferences (Pope, Mays, & Popay, 2007; Suri & 
Clarke, 2009). 
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The qualitative approach is particularly well-suited to the review's purpose and audience because the 
Design Collaborative needs both empirical studies and other literature to identify possible innovations 
for the current education system. An assumption underlying the Foundation's work to fundamentally 
reinvent American education is that the current system fails to deliver college readiness for all 
students, especially Our Kids. This assumption is supported by research indicating that students of 
color and in poverty have low high school and college graduation rates, and research from the last 
two years shows that college graduation rates for minority and poor students have further declined 
(American Council on Education, 2008). Therefore, a priority for the Foundation's work is to identify 
innovations that have not yet been studied, with the intent to evaluate their effectiveness. Literature 
specific to innovations is found outside the traditional scientific or academic journals. 

Inclusive approach. McREL researchers adopted an inclusive approach, searching for and 
including phenomenological reports describing the experiences of Our Kids in and out of school 
and documenting the challenges and successes of their teachers and educational leaders. The 
researchers included literature on innovative, emerging models and untested ideas, as well as 
reports on mature, well-specified models with experimental evidence of effectiveness. Relevant 
quantitative research literature included correlational and experimental studies and meta-analytic 
reviews. Narrative reviews of research were included, as were policy briefs and position papers 
produced by opinion leaders and professional organizations. Literature sources included the World 
Wide Web, peer-reviewed journals, and practitioner magazines. Each document was identified by 
type of literature and evaluated in terms of the quality of the supporting evidence. Care was taken to 
draw only those inferences appropriate to the quality of the evidence. 

McREL researchers judged the quality of the evidence in the context of the type of literature or study 
design and in relation to its relevance to answering particular questions. Guidance from Pope, Mays, 
and Popay (2007) on conducting reviews in the field of health research supports this approach: 

The inclusion of diverse sources of evidence in a review does not mean abandoning the rigor 
of a systematic review, but it does mean judging the quality of evidence in context and defining 
the relevance of evidence to answering specific questions, rather than defining some forms of 
evidence as intrinsically, and universally, of lower quality than others, (p. 1) 

Each research team followed the five or six phases of any review process relevant to a quality 
knowledge synthesis (Cooper, Fledges & Valentine, 2009; Suri & Clarke, 2009). Table 1 (see p. 45) 
provides a side-by-side comparison of the phases of a systematic review of research (Cooper, Hedges 
& Valentine, 2009), a qualitative review (Suri & Clarke, 2009), and McREL's approach to this review. 

Each team began by drawing from pertinent philosophical and theoretical literature and preliminary 
discussions with the Foundation to formulate hypotheses and research questions. Each team 
conducted extensive searches to find as much relevant literature as possible in order to include 
literature from the scientific and academic journals as well as literature from harder-to-find, cutting 
edge innovators. Additionally, teams revisited databases and alternative sources to purposefully 
search for additional literature written by authors identified by one or more stakeholders or to fill 
conceptual gaps that became apparent during the identification and cataloguing of findings and 
generating and communicating recommendations phases. 

The phased process was iterative (Cooper, 2009) re ecting new understanding and insights as 
the search, analysis, interpretation, and discussions between component teams and between the 
Foundation and McREL progressed toward conceptual clarity and the exhaustion of new search hits. 
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Table 1 : Phases of a literature review 


Phase 

Cooper, Hedges & 
Valentine 

(2009, p. 8) 

Suri & Clarke 
(2009, p. 414) 

McREL's approach 

1 

Problem 

formulation 

Drawing from pertinent 
philosophical and theoretical 
discussions 

Identification of key hypotheses 

2 


Identifying an 
appropriate purpose 

Identification of research 
questions 

3 

Data collection 

Searching for relevant 
evidence 

Literature search 

4 

Data evaluation 

Evaluating, interpreting, and 
distilling evidence 

Identification and cataloguing 
of findings 

5 

Analysis and 

Constructing connected 

Generating and 


interpretation 

understanding 

communicating 

6 

Public 

presentation 

Communicating with an 
audience 

recommendations 


The number of documents included in each team's review was extensive, and the types of literature 
varied representing the experiential knowledge of a diverse group of stakeholders, including 
researchers, teachers, administrators, program developers, and leaders and scholars at the local and 
national levels. 

Team approach. Teams were composed of researchers and practitioners with different areas of 
expertise. Teams met weekly, and team leaders from across teams met biweekly. Meetings were 
used to update other individuals and teams and share resources, pose and address questions, 
challenge assumptions, provide guidance on interpretation of evidence, open up new areas of 
consideration, clarify boundaries and overlap between system components, consider alternative 
perspectives, and develop connected understanding. 

Identification of key hypotheses and research questions 

McREL teams began by clarifying terms, relationships, and the conceptual scope of each review. 
Teams read and discussed a document produced during the Foundation's strategy definition process. 
Research Guide for CRLS: Outline of Research Questions for Each Component of the CRLS (n.d.). 
Included in this guide were preliminary questions for each literature review. Teams previewed 
relevant literature, confirmed that the questions could be answered by the extant knowledge base, 
and posed additional questions when important issues related to accelerating learning for students 
of color and poverty were identified in the literature but missing in the guide. The revised set of 
questions for each system component and Our Kids was reviewed and refined during ongoing 
dialogue between the Foundation and McREL. 

Literature search 

Multiple searches were conducted in a phased approach to identify as much literature as possible 
related to each system component and Our Kids. Teams conducted searches using multiple 
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bibliographic databases: Academic Onefile, Academic Search Premier, Educators Reference 
Complete, ERIC, JSTOR, Proquest, and Psychinfo. Teams also conducted manual searches of journal 
and book tables of contents and reference lists of articles. Additional searches were conducted 
specifically to identify recent experimental and other research and reviews on the efficacy of 
interventions for accelerating learning of students of color and poverty. These searches were 
conducted by visiting the U.S. Department of Education What Works Clearinghouse Web site (http:// 
ies.ed.gov/ncee/wwc/reports/) and the Campbell Collaboration Library of Systematic Reviews Web 
site (http://www.campbellcollaboration.org/library.php). Relevant documents were identified on state 
education agency (SEA) Web sites, and SEA officials were interviewed or named as seminal authors 
or sources of models that had been developed and implemented to monitor and accelerate learning 
of Our Kids. 

Each team identified and used key terms and synonyms relevant to the topic for searching. Searches 
were conducted for literature published in the most recent 10 years (1998-2008); however, works 
by seminal authors and other recommended literature were included from outside these years. The 
search landscape varied for each team based on the topic and relevant sources; for example, while 
What Works Clearinghouse was a relevant source for the Pedagogy team, it was not a relevant 
source for the Leadership/Human Capital team. Internal review of search records and results led to 
additional leads on sources. Searching continued until all recommendations had been implemented 
and/or few new hits were identified. 

Identification and cataloguing of findings 

A coding protocol was developed and implemented to categorize the literature. Each team used the 
same protocol, adding categories and decision rules, as needed to organize the particular literature 
relevant to their topic. Each team leader and one or more members of each team were trained on 
the decision rules in the coding protocol and provided follow-up support to resolve uncertainties in 
its application. Team leaders periodically conducted quality assurance reviews of completed coding 
sheets and updated the protocol as needed during weekly team leader meetings or discussions with 
the Foundation. The coding protocol included identifying the following information: 

• Full APA reference citation 

• Category of literature (i.e., primary and secondary relevance) 

• Type of literature (e.g., quantitative study, policy brief, program description) 

• Locale 

• Outcome 

• Grade level 

• Program or innovation name and description 

• Main findings or points 

• A recommendation for or against summarizing and including the selection in an annotated 
bibliography. 

In addition, component teams added to the protocol by categorizing relevance to particular parts of 
their conceptual model or concept map. 

Guidelines were developed and used by teams to identify counterproductive orthodoxies, unmet 
needs, next practices, promising practices, and best practices based on type of literature and quality 
of evidence. These were defined in the following ways: 
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• Counterproductive orthodoxies: Conventional ways of providing education which may be 
impeding success of Our Kids 

• Unmet needs: Areas where Our Kids are not yet well served by the current system of education 

• Next practices: A program or practice that needs to developed, adapted, invented, and tested in 
response to an unmet need related to accelerating learning for Our Kids 

• Promising practices: Practices based on research but not supported by rigorous efficacy data 
from randomized controlled trials 

• Best practices: Practices demonstrated by one or more randomized controlled trials to be 
effective in improving outcomes for Our Kids 

The research team reviewing the college readiness component of the Learning System employed a 
slightly different process. Rather than using the categories above, this team reviewed literature on 
college readiness and categorized findings into four essential areas as defined by the Foundation and 
Conley (2007): cognitive strategies, content knowledge, academic behaviors, and contextual skills. 

Component teams met weekly to discuss and categorize findings and to develop a conceptual map 
of the insights gained from the literature summaries and review. Teams used different conceptual 
mapping tools (e.g., SmartArt) to organize the insights (findings) and presented and discussed 
their respective maps at cross-team meetings. Features common across teams' concept maps were 
identified and a standard framework developed. Teams arranged findings onto the concept maps, 
identifying conceptual gaps and con icting or discrepant findings, and returned to searching and 
reviewing to fill in the gaps and resolve or explain discrepant findings. The conceptual maps served 
as an organizing framework for report construction. 

Generating and communicating recommendations 

Working collaboratively, component teams drew conclusions from the insights (findings) derived 
from the review and identified potential options and recommendations for each component 
of the system. Teams used an iterative process of identification, reviewing for validity against 
the knowledge base, and further refinement until they determined they had identified the most 
promising options and that each was informed by the existing knowledge base. 

Team leaders used the outcomes of team discussions and cross-team discussions, literature 
summaries, and the researcher's own review and integration of the literature to write a draft report of 
the findings. Draft reports were reviewed by knowledgeable internal experts and revisions in search 
strategies, interpretations of findings, and/or conclusions were made. Revised reports were reviewed 
by the Foundation and other outside reviewers prior to final revisions and production. 

Although the wide-ranging literature searches produced reports on extensive baseline information 
related to Our Kids and each system component, the reports are living documents. As living 
documents, they bridge the creative and scientific enterprises of the past and present, and we 
envision the need to return to some of them for updating, extending, and drilling-down in the future. 
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